We provide concrete rules below for the two competition tracks that comprise DATACOMP: filtering and BYOD . Additionally, we provide a checklist, which encourages participants to specify design decisions, which allows for more granular comparison between submissions. A.1 Filtering track rules Participants can enter submissions for one or many different scales: small, medium, large or xlarge, which represent the raw number of image-text pairs in CommonPool that should be filtered. After choosing a scale, participants generate a list of uids, where each uid refers to a COMMONPOOL sample. The list of uids is used to recover image-text pairs from the pool, which is used for downstream CLIP training.

Attention over learned object embeddings enables complex visual reasoning.

AAdditional Experiments620 Task 1 - Grouping In addition to grouping clue words using token embeddings (discussed in621 the main paper 4), we also ran grouping the words by clustering on'contextual' embeddings. We622 experimentally induce'context' by joining the sixteen (16) word tokens (in a random order) into a623 single pseudo-sentence. The embeddings for each token were different based on the ordering of the624 tokens. We repeat the random ordering sixteen times and report the mean and variance of the results625 obtained in Table 6.626 Mean standard deviation over 16 random seeds is shown. Task 2 - Connections In addition to prompting based results on GPT-4 (discussed in 4), we ran627 experiments on additional LLMs like LLaMa [67] (7B, 13B) using pre-trained configuration weights628 obtained by permission from Meta AI. However, without additional fine-tuning on the specific task,629 these LLMs were unable to solve the task in a meaningful manner.

In this supplementary material, we present additional details of the proposed Flare7K dataset and experimental settings and show more results. Figure 1: Illustration of a simplified lens system. In the lens and aperture plane, the light passes through the dirty aperture and lens system, leaving a scattering flare on the image plane. In this section, we use a simplified Fourier optics model to illustrate how different kinds of scattering flares occur. A basic lens system can be viewed as a combination of one convex lens, one aperture, and an image plane as shown in Figure 1. We set the optical center as the origin of a coordinate system. Then, the light source's position is (x0,y0, z0). It is a combination of aperture function eAλ(x,y) and a lens function eTL(x,y). Supposing the focus of the lens is f and the lens is ideal. After adjusting the origin of x1 and x2, Equation (11) can be viewed as a standard Fourier transformation. Thus, the point spread function (PSF) which is the square of the amplitude of the image plane's optical field can be written as: PSFλ = |F{eAλ(x,y)}|2. Since stains with depth may bring phase shift for the aperture function, the PSFλ may vary with the wavelength λof the light source.